Text Recognition and Translation of Multi-Oriented, Multi-Language and Curved Text in Natural Scene Images

This study is about text detection and recognition in natural scene images. The main focus is on the detection, recognition and eventually, translation, of multi-oriented, multi-language and curvilinear text in such images. The study attempts to provide a solution that can detect and recognise such text since current leading mobile applications such as Word Lens and Google Goggles do not support such text for translation. There are many algorithms available that can detect and recognise text, but very few consider text which is multi-oriented, multi-script or curvilinear. Text detection can be carried out using various methods including region-based and texture-base methods. Furthermore, algorithms for multi-oriented text detection are further divided into non-headline and headline based methods. Three different solutions were considered in this study, one being an algorithm developed specifically for maps in which text usually has various orientations, curvatures and sizes. Another option was a framework that performed detection and recognition simultaneously. The third option was a combination of two algorithms, one for detection and one for aligning curved and multi-oriented text. The research carried out consisted of implementing the final option and integrating these two algorithms to achieve a system that could detect and recognize multi-oriented, multi-script and curvilinear text. Tests were carried out using the proposed system by using different data sets that are publicly available for testing such systems. Results were recorded according to the criteria of time, precision and recall. In addition, the proposed system was compared to two leading applications: Word Lens and Google Goggles. The result of this study was that the resulting system was not capable of performing detection and recognition of all kinds of images and text. On the other hand, it was able to give a solution for issues that other leading applications face, such as the detection of multi-oriented and multi-script text. Results can be affected by various aspects, including the libraries and languages used to implement an algorithm. In addition, the device used and its processing power are highly related to the performance and probably also to the precision of a system. Other factors that need to be considered when implementing such a system are the datasets that will be used for testing and the experience and knowledge one has in the area.